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ABSTRACT 

The lEA Study of Writing was preparc^d in response to 
an increased concern for an interest in world literacy. Additional 
issues addressed were: (l) setting learning in the context of the 
cultural framework, curricular emphases and teaching practices, 
rather than just determining the level of achievement, and (2) making 
it possible for each participating country /school system to assess 
its relative strengths and weaknesses in writing instruction against 
an international background. Papulations for the stuay were chosen 
from 3 stages of schooling and from 14 countries. Each student in the 
sample populations wrote at least 3 compositions from a total corpus 
of 14 different task versions. The compositions were supplemented by 
background information supplied in questionnaires completed by 
studcints, teachers, and school administrators. The rationale for the 
instruments used and the expected results of the study are outlined. 
(JAZ) 
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ABSTRflCT 



INTEBNATICWAL study CF WRTl'ING: WHY AND KOW? 

R. Elaine Degenhart 
and 
Sauli Takala 

Ihej lEA International Study of Writing was inaugurated in response 
to an increased concern and interest in world literacy. Ihe inportance 
of writing as well as reading in society has gained recognition in a 
recent inovemsnt toward a greater balance betmen the twD forms of literacy 
in education, ihis study addresses this issue with the additional pur- 
poses of (1) setting learning in the context of the cultural framewDrk, 
curricular ernE^iases and teaching practices, rather than just determining 
the level of achievement, and (2) making it possible for each participating 
country /school system to asr^ss its relative strengths and weajuiesses in 
writing instrjction against an international background. 

Ihe study consists of fourteen countries f each testing one to three 
age groups {f>opulations) . Ihe students in the saitple have each written at 
least three cor^csitions fron a total corpus of fourteen different task 
versions. Ihese coc^sitiois (the dependent variable) were suppleirented 
by background inforrration si^plied in questionnaires conpleted by the 
students, teachers, and school administrators involved in the study (the 
independent variables) . 

Ihis paper outlines the purpose of the study, the rationale for the 
instrunentii used, and the expected results. 
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INTERNATIONAL STUDY OF WRITING: WHY AND HOW? 

R. Elaine Degenhart 
University of Illinois and 
Institute for Educational Research, Jyvaskyla 

and 

Sauli Takala 
Institute for Educational Research, 
University of JyvkskylS, Finland 

With the growing awareness of the importance of li^^eracy, composi- 
tion, perhaps more than reading, has become a focal point for critics of 
schools. Writing, for obvious reasons, is one of the most visible 
products of edj^cation. Incorrect usage and spelling have been inter- 
preted as signs of personal scholastic failure, and the alleged wide- 
spread deterioration of writing ability as an indication of inadequacies 
in whole school systems. In view of the importance of writing in society 
and in the educational system, it is not surprising that some countries/ 
school systems have begun to assess systematically the effec iveness of 
the teaching and learning of writing. 

The study reported here was mounted to accommodate this inter- 
nationally strengthened interest in the assessment of writing. This 
reflects the recognition of the central plaoe that the study of the 
mother tongue occupies in the school curriculum. Introducing students 
to written language, and thus promoting literacy, has traditionally been 
perhaps the principal task of the school. Learning to read has always 
been emphasised, while writing has tended to receive somewhat less 
attention. Recently there has been a movement toward a greater balance 
between the two forms of literacy. The lEA study seeks to accommodate 
this interest, with two additional purposes; (1) to set learning in the 
context of the cultural framework, curricular emphases and teaching 
practices, rather than just to ascertain the level of achievement, and 
(2) to make it possible for each participating country/ school system to 
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assess its relative strengths and weaknesses in writing instruction 
against an international backdrop. 
Who participated in the study ? 

This is a study of the teaching and learning of written composition 
in the schools of fourteen countries/school systems; Chile, England and 
Wales , Federal Republic of Germany (Hamburg) , Finland , Hungary , 
Indonesia, Italy^ Netherlands, New Zealand, Nigeria, Sweden, Thailand, 
and United States. Several other countries have participated in various 
phases of the study, but for a number of reasons were not involved in 
the full range of its activities, which included curriculum analysis, 
pilot testing, main testing, and data analysis. 

Three stages of schooling were seen as significant points for 
students ana became the pcpulations tested in the study. Population A 
is defined as students at or near the end of primary education and the 
self-contained classroom. Pi^pulation B consists of students at or near 
the end of compulsory education. Population C comprises students at or 
near the end of the academic secondary school. For Populations A and C 
there were a minimum of fifty schools in each country's sacaple, and for 
Population B each country sacapled at least one hundred schools. 
What are some specific differences in a study of writin g? 

One of the most important considerations the project bad to deal 
with was the fact that writing differs, for example, from mathematics 
and science, in that the criteria of what is the correct or at least a 
good response may vary somewhat from culture to culture. It would be 
presumptuous to maintain that there is only one single correct product 
as a response to a typical composition task. A typical characteristic 
of all composition tasks (not only in mother tongue instruction but in 
many other subjects as well) is that there are several acceptable 
approaches and several acceptable products. 

H 
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There tends to be a fair degree of agreement within certain 
cultures, which are sometimes called interpretive communities, but 
cultures laay differ in terms of 

a) what functions of writing are emphasized in school; 

b) what patterns of organization (style, rhetoric) are preferred and 
rewarded; 

c) what are appropriate topics to write about* 

d) what is the appropriate approach to writing (e.g., personal vs. 
impersonal, serious vs. humorous); 

**) what is the appropriate form of task instruction (e.g., mere title 
vs. detailed prompting); 

f) what is the appropriate time to allow students to write in response 
to an assigriment; 

g) what are the appropriate criteria for rating compositions. 

This essential cultural relativism is at the same time one of the 
major problems of the project and one of the most interesting and 
challenging aspects of the study. A study like ours seems to address 
central areas of comparative education. On the one hand* it can be 
asked if cultural differences are so large that valid comparisons 
between student writing are at all I'easible. Very little is known of 
cultural patterns and preferences in this area. For this reason we were 
prepared to be content with fairly modest results. What we hope to be 
able to do is to get a better idea of the differences and provide good 
descriptions of them. 

On the other hand, since not much is known, it can also be asked to 
what extent there actually are distinct cultural patterns in writing or 
whether such a notion is a stereotype baised on anecdotal evidence. Is 
there some general pattern as well as variation within each culture and 
to what degree is there overlap? 
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Another major problon for the study is the fact that in most casos 
students are writing in their mother tongue but in gome countries they 
will be writing using the language of instruction, which may not ba the 
mother tongue. 

Thus we have concluded that the written composition study differs 
from other current studies quite clearly through the fact that (a) the 
response of each task is not fully definite at the outset but becomes 
more definite during the process of writing, (b) there is a large store 
of potentially applicable knowledge, and (c) there is a fairly wide 
choice of alternative routes to the goal. 
What audience will the study serve ? 

In planning the study, the Steering Committee was faced with the 
fact that there was very little international cooperation in the field 
of mother tongue teaching, and that most participating countries lacked 
good accounts of writing practices and of students'^ writing performance. 
In addition, there seemed to be a clear need for defining the domain of 
writing as a starting point of the study. Finally, it would be necessary 
to develop a procedure which WDuld make it possible to rate compositions 
written by students in several countries and in several different 
languages in as comparable a manner as possible. 

Given this situation, the Steering Committee of the project 
considered mainly three alternative approaches. Approach A would focus 
on the classroom level by gathering detailed data on teaching practices 
and attempt to determine whether some practices emerge as consistently 
effective in several school systems. Approach B would place the 
emphasis on writing tasks in order to survey the performance of a repre- 
sentative sample of students, and various sub-group;, on a good sample 
of tasks. In Approach C a great number of background variables pertain- 
ing to national economic, social and educational systems as well as 
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school and classroom variables would be measured and related to student 
achievement. 

Each approach was aimed at providing information of particular 
interest to a specific audience. The primary audience of Approach A was 
considered to be curriculum planners and teacher eaucators. In addition 
to Audience A, Approach B would be attractive also to linguists, 
especially to scholars interested in cross-cultural rhetorics. Approach 
C would be of special interest to policy-makers and students of educa- 
tional policy and systems, who concern themselves with the output of 
educational systems in the light of social, economic and educational 
factors. 

In practice, it turned out to be difficult to reach a clear con- 
sensus on the priority of any of the three approaches. Therefore, as a 
compromise, data collection was planned in such a way that, at some 
stage of analysis, the data allow some answers for all three audiences. 
However, as work on the project progressed. Approach C seemed to lose 
much of its appeal for the group of researchers involved in the study. 
So little is known about the teaching of writing and about student 
perforEoance in writing that Approaches A and B emerged as the first 
priority. Answering such questions first seemed to be necessary before 
attempting to answer system-level questions about factors that might 
possibly be related to writing performance. 

Consequently, it was concluded that the study would make the most 
meaningful contribution (a) by providing a good assessment of how well 
the school systems have succeeded in helping students to become flexible 
writers, i.e., to write competently on a number of d\fferent tasks, (b) 
by producing good descriptions of how composition is taught in schools, 
(c) by trying to tetermine whether certain teaching practices contribute 
to good learning outcomes in several school systems. 

7 
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Since this subject area is strongly influenced by cultural 
characteristics, it was apparent that the national results would have to 
be interpreted, first and foremost, by each National Center. This has 
been strongly emphasized as w*^.ll as the need to involve the national 
communities of writing instruction and writing research in the planning, 
conduct and reporting of the study. 
What did we wish to accomplish ? 

With this cultural and research context in mind, the lEA Inter- 
national Study of Written Composition was dei-igned to accomplish the 
following tasks: 

(1) To contribute to the conceptualization ox* the domain of writing and 
particularly the domain of school-based written composition; 

(2) To develop an internationally appropriate set of writing tasKs and a 
system for assessing compositions which is applicable across countries/ 
school systems and across languages; 

(3) To describe recent developments and the current state of instruction 
in written composition in the participating countries /school systems; 
(^) To identify factors which explain differences and patterns in the 
performance of written composition and other outcomes, with particular 
attention to cultural background, curriculum and teaching practices} and 
(5) To make a contribution toward solving problems related to the 
assessment of essay-type answers, particularly when more than one 
language is involved. 

How was the dependent variable dealt with? 

The greatest challenge in the study has been the work cn the 
dependent variable, since writing assignments and expected writing per- 
formance might vary from one school system to another. In order to 
unaerstand and compare the approaches taken in different school systems, 
we needed a systematic account of the domain of school writing. In this 
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regard, three problem areas had to be addressed: (1) the construction of 
writing tasks, (2) the allocation of writing tasks, and (3) the rating 
of student scripts. 

As a first task in addressing these problems, the key appeared to 
be in a moi'e dc^tailed specification of the domain of writing. Accord- 
ingly a system was developed based on a theory of written discourse. In 
this system the primary purpose and audience of writing is related to 
the level and object of the cognitive processes involved in the writing 
vask . 

Oil the basis of this domain specification system, and after a 
careful scrutiny of the writing curricula and typical writing tasks/ 
topics used in the participating countries, nine different tasK types 
containing 1^1 different task versions were developed. These tasks 
emphasized several important perspectives. 

(1) Tasks that emphasize the perspective of the writer 

Task 5t write a personal story 

Task 8: write a "free** composition on an ambiguous and evocative 
pictorial stimulus 

(2) Tasks that emphasize the perspective of the topic 

Task 2: summarize a text 

Task 3: retell a story (in a shorter form) 

Task i|a: describe a ritual mask 

Task ilb: describe a process of doing something 

Task 7: write a reflective essay 

(3) Tasks that emphasize the perspective of tne reader 

Task 6: try to persuade the reader to share the writer's strong 
view about something 
(^) Tasks that have several perspectives 

Task la: describe a desired bicycle to an uncle who wishes to buy 

6 
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one as a birthday present 
Task lb: describe oneself to a penfriend whom th'? student is going 

to visit so as to make it possible for the penfriend to 

identify the student as he comes to meet the student 
Task 1c: write a note to the principal/heaamaster cancelling a 

scheduled meeting 
Task Id: leave a message at home telling where the student has gone 

after school 

Task 1e: write a letter applying for an advertised summer job 
(Note that the relative writer-reader social status and the topic 
is varied systematically within the variations of Task 1.) 
Task 9: write a letter to a younger student who is coming to study 
at the same school as the writer', telling the new student 
how he/she should write in the new school to get good 
grades. 

The samplin^i of tasks is more comprehensive than is typical in 
writing assessments. However, tasks were rotated in classes so that each 
student wrote on only 3 (sometimes H) tasks, which differed in terms of 
writing stimiCus. After some negotiation and revisions based on pilot 
testing, there was a high agreement that the final sets of tasks 
provided a fair measure of students' writing ability in all partici- 
pating school systems. 

Within this agreement, however, it was recognized that different 
cultures may have somewhat different views of how to produce discourse 
for different purposes and in different contexts. Tnus the.'^e may be some 
variation in the way students in the participating countries write about 
the lEA tasks. However, it is assumed that raters in participating 
countries can, after discussion and training, come to share a largely 
similar view of the merits of compositions, in spite of potentially 



/ 



AERA, San Francisco, April 16-20, 1986 



9 



ERIC 



different ways of discourse production. Unlike a subject like mathema- 
tics and science, rating involves judgement. However, as in the fine 
arts, raters can develop their sensitivity to merit and write a critical 
comment on each composition. The empirical verification of this basic 
assumption through careful checking of thoroughly trained raters was an 
essential condition for an evaluation study which presupposes that 
scoring is done in a sufficiently uniform manner in all participating 
countries. 

After considering and trying out various scoring systems {holistic, 
primary trait, analytical)^ it was agreed that the most appropriate one 
combined a holistic overran impression marking and analytical marking* 
Writing competence within this system is understood to consist of 
"discourse-structuring competence" (or rhetorical competence) and of 
"text-producing competence*'. Rhetorical competence is assumed to require 
both "cognitive competence" and "social competence" . Cognitive 
competence is manifested in the ability of writers to make readers 
easily recognize the communicative intent. Text-producing competence 
presuppose:; ^'linguistic competence" and "motor competence", which are 
rated in accordance with national norms. 
What were the independent variables? 

A major aspect of the study has been a comprehensive analysis of 
the context cf written composition teaching in the participating coun- 
tries* In order to provide this context, each country provided a set of 
ratings indicating to wha:. extent certain objectives (content areas and 
processes) are covered in the curriculum at each population level* These 
ratings, collected by means of a Curriculum Questionnaire, were used as 
a guide in preparing both the tes»:s and questionnaires and will be used 
in interpreting results* In addition, this data was supplemented by a 
national Case Study, which outlines in particular (a) national literary 

/I4 
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developments, (b) schools of thought with regard to uiother tongue educa- 
tion, (c) the position of teachers and education, (d) examinations and 
assessment procedures, (ft) minimum competency movements, and (f) writing 
outside of school. Using an Interview Schedule developed by the Steering 
Committf^e, each of the National Research Coordinators gathered data from 
a number of national experts which was then complemented by reviews of 
written documents. Sucti case studies are considered ui^eful products in 
themselves, but they are also expected to be of great use in the inter- 
pretation of the results. 

Of particular interest in the area of school writing context are 
the portraits of writing instruction that the very specific Teacher 
Questionnaire data will provide. This questionnaire was given to all the 
teachers who teach written composition to the students in the samples in 
order to obtain information on the teachers' qualifications, experience, 
teaching and feedback methods, etc. It is with this information that the 
study hopes to find out what teaching practices might be related to good 
writing performanca. A School Questionnaire was answered by the school 
principals and provides data on the community and the school. 

All questions in the Student and '"eacher Questionnaires were based 
on the theoretical moo**! developed for the study, and therefore, pre- 
sumably are variables of potential descriptive and explanatory interest* 
As a general principle, countries were also encouraged to ddd other 
types of national options to the basic international components, 
provided that the latter is not Jeopardized. As a result of this careful 
planning, the total testing program in any gjven country is perceived as 
appropriate by students, teachers and other interested parties, and 
provides information that is relevant in terms of national problems in 
the instruction of written compcsition- 

4^ 
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What results can be expected ? 

The scoring scheme was tested for the first time during the th'^rd 
International Study Committee Meeting in Urbana in the spring of l98ii. 
The results of this scoring session were thoroughly discussed and it was 
concluded that in spite of a great effort, the scoring metric is too 
elastic to all^w robust cross-national comparisons of scores on any one 
task or groups of tasks. For anyone working in the assessment of written 
composition, the results in an internationsxl attempt to create a fully 
comparable scoring procedure are not surprising. In fact, we hsve been 
3omewhat surprised that we have been able to solve as many problems as 
is the case. 

While robust comparisons of achie^vement across countries are 
doubtful, we do not consider that we have failed in our attempt. It 
appears that tasks 1, 2, and 3 of the study design outlined above will 
be successfully accomplished. We have specified the domain of school- 
based writing; we have developed an internationally appropriate set of 
writing tasks; and we are providing descriptions of the recent develop- 
ments and the current state of instruction in written composition in the 
participating countries and school syster^* That is more than some of us 
had dared to expect. 
How will the project be reported ? 

Three international reports have been planned. The first report is 
nearing compilation and presents the development of the tasks wiUh 
comments on their construct and currlcular validity; the development of 
the scoring scheme with comments on its construct validity; and the 
presentation of the international rating scale with comments. The second 
report will olace the various depictions of writing instruction within a 
larger cultural framework using the data gathered with the various 
questionnaires and the National Case Studies. The third report will 
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focus on the dependent variable and report on the scoring in each educa- 
tional system, the types of meaningful scores that can be compared, and 
the results of the approved comparative exercises. 

In addition to the international reports, each participating 
country is expected to write a major national report presenting the 
national results, preferably in the international perspective. National 
Centers are encouraged to supplement these reports with shorter publica- 
tions making the maximum use of the data and providing the greatest 
possible dissemination of the study results. 

For such a comprehensive study, this has necessarily been only a 
very brief introduction to the many possible directions for analysis and 
interpretation of a massive data base on a very complex and vital 
subject. We only hope that we have been able to excite your curiosity 
and to whet your appetite for further research and reports in this area. 
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